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ABSTRACT 

The purpose of this study is to examine the effects of reflective thinking effects in the process of designing 
software on students’ learning performances. The study contends that reflective thinking is a useful teaching 
strategy to improve learning performance among lower achieving students. Participants were students from two 
groups: Higher achieving students were the control group, and lower achieving students were the experimental 
group. The experimental group students wrote weekly diaries demonstrating reflective thinking in the processes 
of designing software. The study’s results show the evaluation scores of the two groups of students’ software 
designs were comparable. The abilities of the experimental group to comment on a design improved. This shows 
that a reflective thinking strategy did have positive effect on improving lower achieving students’ learning 
performances, especially in the process of software design. Discussion of the results and suggestions for future 
study appear at the end of this study. 

INTRODUCTION 

Currently, numerous software design tools are helpful for rapidly constructing an environment of digital learning, 
which contributes to achieving expected learning goals and efficacy. However, when considering learners’ 
backgrounds, software developers should ponder principles of accessibility and usability, to integrate learning 
theories into designs (Hackctt & Parmanto, 2006; Hsieh, 2008; Kidney, Cummings & Boehm, 2007; Lam, Lam, 
Lam & McNaught, 2009; Powell & Murphy, 2009; Zhang, Zhang, Duan, Fu & Wang, 2010). For example, when 
a web designer develops a search engine, an invitation to users allows participation and exploration of behavior 
and cognition simultaneously, and application of a learning theory should be used to clarify issues of usability 
and accessibility in web design. Subsequently, the web designer can revise the original design’s concepts and 
finalize the search engine’s design. 

According to Dewey (1933) reflective thinking is, “the kind of thinking that consists of turning a subject over in 
the mind and giving it serious and consecutive consideration” (p. 3). The software design process requires 
reflective thinking, in addition to ordinary thinking; namely, previous thoughts and conduct should be reviewed 
through “reflective thinking,” prepared for deep learning (Dewey, 1933). If time for reflective thinking is 
available during the process of software design, capitalizing on opportunities for undertaking reflective thinking 
involves acquiring descriptive content (or facts), procedural knowledge (conceptual or induced from formulas), 
and high-level learning (i.e., deep learning). An example of reflective thinking process is: Teachers encourage 
students demonstrating low learning performance to access previous learning, revisit techniques employed to 
gain the learning, and estimate knowledge deficits requiring fulfillment. This reflective thinking process has the 
potential to improve performance and unique personal creativity of low learning students (Schunk, 2004). 

According to Social Cognitive Theory, social environments and personal beliefs or cognition affect the ways 
humans thinking. In other words, “self-efficacy” (i.e., an individual's self-confidence level for completing a 
special task) will be different, depending on different situations (Bandura, 2002). Some studies’ empirical results 
indicated that students having higher levels of self-efficacy achieved higher learning performance, and vice versa. 
Since students gained confidence from positive experiences of achieving high learning performance, their high 
levels of self-efficacy further advance (McMurray & Sanft, 2005; Phan, 2007, 2008). In addition, a few studies 
examined whether or not students involved in reflective learning processes attained significant benefit in 
learning performance, and their reflective learning outcomes influenced their levels of self-efficacy. Therefore, 
the current research designs an experimental study to verify whether or not reflective thinking effects exist for 
students in a software design course, especially those demonstrating lower achievement. Overall, this study’s 
three core questions are: Do lower achieving students’ learning performances improve from reflective thinking 
processes? Do lower achieving students’ self-efficacies for computer use enhance due to the effects of reflective 
thinking? Finally, through exploration what reflective thoughts are produced and then discovered from lower 
achieving students’ perspectives for a commenting task? 
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LITERATURE REVIEW 

Clarification of this study’s purpose requires discussion of the meaning of learning through reflective thinking 
and the causal relationship between reflective thinking outcomes and individuals’ levels of self-efficacy. The 
literature review also includes the design principles and the multi-dimensional assessment criteria adopted in the 
software design course for implementing learning through reflective thinking. 

Learning through Reflective Thinking 

The conceptual definition of the term, reflective thinking, originates with a proposal from the scholar John Lock 
in 1690, and subsequently the scholar John Dewey transformed this concept into an operational principle, which 
asserts that reflective thinking can result in true, purposeful, and meaningful learning (Dewey, 1933). The 
definition of reflective thinking suggests that the process can result in great value, despite the effort required. 
The process of reflective thinking affords personal creativity, meaning, and criticism from learning activities. 

In terms of exploration of reflective learning, the process of learning encompasses deep or high-level learning, 
which means engaging in critical thinking, obtaining cognitive and metacognitive awarenesses, operating with 
sophisticated conceptual thinking, and originating creative ideas to solve problems (Chang & Chou, 2011; 
Hatton & Smith, 1995; McCrindle & Christensen, 1995; Thorpe, 2004). In other words, reflective learning 
transforms and re-digests acquired knowledge to solve problems and demonstrate personal creativity - perhaps, 
highlighting personal values during knowledge acquisition and integration. 

Prior study results indicated the positive impact of reflective thinking on learning achievement (Ersozlu & 
Arslan, 2009; McCrindle & Christensen, 1995; Sheorey & Mokhtari, 2001). Those results showed either 
significantly improved scores obtained by experimental groups or obviously comparable learning performance in 
the assigned learning tasks when compared other students in control groups. In addition, as an individual learner 
proceeds through a reflective thinking process, awareness of strategies for learning may simultaneously enhance. 
The learner’s perception of level of ability to learn (i.e., self-efficacy) increases as well. The McMurray and 
Sanft (2005) study also indicated a strong correlation between reflective thinking and self-efficacy. The Phan 
(2007) study empirically verified the causal effects of reflective thinking and self-efficacy on academic 
performance. The importance is the necessity of encouraging students to think reflectively during learning 
processes. 

At present, one of the most commonly seen avenues for learning through reflective thinking in classrooms is 
practice through projects. Students, individually or in groups, can learn from solving case problems, learn from 
doing, and do from learning. Also, students can actively participate in other activities before, during and after the 
class (e.g., preview, review, and discuss learning content), to enhance the content quality of reflective thinking 
and then achieve the expected learning results (Thomas, 2000). As for other ways of learning through reflective 
thinking, the literature suggests that teachers adopt different teaching strategies: completing reflection sheets, 
writing a reflections diary, storytelling, or debating openly (Chen, Kinshuk, Wei & Liu, 2011; Hatton & Smith, 
1995; McKillop, 2005). The current study asks students to write reflection diaries, and in deference to today’s 
high technology, pen and paper are not necessarily the only technique for recording. Instead, student wrote 
personal reflections of learning on blogs (Efimova & Fiedler, 2004) and had the freedom to add representative 
images and pictures to supplement their reflections. 

Self-Efficacy 

Based on the perspective of Social Cognitive Theory, “self-efficacy,” proposed by Bandura is a learner’s 
cognition of self-confidence and the capability to achieve a degree of competence in a specialty (Bandura, 2008). 
It is an individual’s evaluation of self-confidence and belief in ability to accomplish a mission (Schunk, 2004). 
Self-efficacy is different from ability. With expected outcomes, self-efficacy is a person’s cognition of ability 
when taking action. Cognition changes, as exposure and retention of information expand (Bandura, 1986). For 
example factors, such as personal emotional change, anticipation for results, awareness of others’ expectations, 
previous experience, environmental conditions, and so on, can affect an individual’s self-evaluation of ability. 
Among the factors, arguably, previous experience is a strong predictor of self-efficacy, because previous positive 
or negative learning experience influences the extent of a learner’s evaluation of self-efficacy. Those who had 
positive learning experiences gain enhanced self-efficacy more easily than those who had negative learning 
experiences. In addition, a learner’s self-efficacy will likely change during different times, and situations, and 
build inner value for cognition for self-ability (Bandura, 2002; i§man & Qelikli, 2009; Topkaya, 2010). 

For more than 20 years after 1977, much research of self-efficacy appeared, including correlations, pre-test and 
post-test comparisons, and experimental studies. These studies mainly explore the influence of self-efficacy on 
learning achievement, including: solving conceptual math problems, writing, and reading abilities, learning 
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motivation (Bong & Clark, 1999; Schunk, 1991; Zimmerman & Bandura, 1994; Zimmerman, Bandura & 
Martinez-Pons, 1992), occupation choice or career decision, etc. (Betz & Hackett, 1981; Chaney, Hamoond, 
Betz & Multon, 2007; Hartman & Betz, 2007). The literatures, suggests that self-efficacy development 
encompasses many patterns. Self-efficacy mentioned in various studies circumscribes the same concept, but 
within differing contexts (Multon, Brown & Lent, 1991; Pajares & Miller, 1994; Schunk, 1991; Zimmerman, 
1995; Zimmerman & Bandura, 1994; Zimmerman, Bandura & Martinez-Pons, 1992). For example, self-efficacy 
in schoolwork is the student’s cognition of ability for achieving goals in learning. Self-efficacy in teaching is a 
teacher’s cognition of an ability to help students learn, and consequently influences motivation and willingness 
to prepare instructional materials and activities. Self-efficacy in computer technology is a self-rating ability to 
apply computers to diverse situations, such as searching information via the Internet and using multi-media 
software. Consequently, questionnaires developed for different studies for measuring self-efficacy in various 
contexts are unique (Hsieh, 2009). 

For measuring self-efficacy of computer use, the General Self-Efficacy Scale (GSE), developed by Schwarzer 
and Jerusalem is the most common questionnaire, which is able to predict a student’s performance with 
computers in daily life, under pressure, and so on (Schwarzer & Jerusalem, 1995). However, according to studies 
conducted by Joo, Bong and Choi, a questionnaire’s design, which has high credibility for measuring 
self-efficacy for Internet use merely measures a student’s ability to search web pages effectively, not a more 
general ability for learning (Joo, Bong & Choi, 2000). Thus, instead of utilizing GSE, the Thatcher and Perrewe 
(2002) Computer Self-Efficacy Questionnaire is an alternative for determining students’ abilities to search web 
pages. However, to measure students’ abilities of packaged software using, this questionnaire maybe not an 
appropriate one. 

This study adopts the Compeau and Higgins (1995) Computer Self-Efficacy Questionnaire, researched and 
designed by Compeau and Higgins (1995). The questionnaire contains ten Yes/No questions and measures both 
the magnitude of self-efficacy, so it can provide data of students’ abilities to use packaged software. Compeau 
and Higgins established the reliability and validity of analyses obtained from this questionnaire after using 
Partial Least Squares to test a research model. The results of their analyses showed the questionnaire’s high 
internal consistency (reliability coefficient greater than 0.8), and strong construct validity (higher than 0.7). Thus, 
to examine whether or not lower achieving students, who had undertaken learning through reflective thinking, 
would demonstrate significant change in self-efficacy, after completing an animation software design assignment, 
adoption of this questionnaire is more useful and appropriate than others. 

Design Principles and Multi-Dimensional Assessment Criteria 

Arguably, design principles, such as accessibility and usability, are fundamental skills for novice designers. 
Other design principles, such as aesthetics, attention-getting, friendly, responsiveness, simplicity, error tolerance 
and reliability, are also software designers’ considerations. Also, design principle may have application in some, 
but not all, design projects. For example, novice web designers must execute several accessibility checks, 
including HTML elements, multimedia elements, web tools, and advanced scripting (Institute for Interactive 
Technology, 2006), to ensure the accuracy of the information presented on each web page; usability checks are 
likely necessary as well. For novice software designers, prior to the processes of software design, all usability 
checks are important considerations, including: compatibility, consistency or coherence, directness 
(WYSIWYG-what you see is what you get), interactivity, and user-in-control (Nielsen, 1994). 

Expert software designers intuitively consider most design principles in their software development processes, 
without consciously applying design principles, they are aware of most users’ preferences and update designs to 
comply with the latest trends. The most effective way to achieve most design principles’ requirements is to invite 
software users entering the processes of software design to test software beta versions, even though such testing 
is sometimes very time consuming (Ropinski, Meyer-Spradow, Steinicke & Hinrichs, 2006). Then software 
designers and users have chance to communicate with each other during development before releasing the 
software to the market. This approach, called user-oriented design, is increasingly popular among both novice 
and expert software designers. Overall, by taking design principles into consideration or taking user-oriented 
design approach, designers have gone through a reflective thinking process during the process of design. 

In order to objectively assess students’ performances for designing animation software, this study adopts 
multi-dimensional assessment criteria provided by the Association to Advance Collegiate Schools of Business 
(AACSB, http://www.aacsb.edu/). In recent years, AACSB has supervised many educational institutions to 
establish multi-dimensional assessment criteria to ensure quality of global business education not only in the 
United States, but also around the world. The academies achieving the criteria’s quality standards gain 
certification from AACSB (http://www.aacsb.edu/accreditation/). This study evaluates students’ learning 
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performances according the AACSB’s five criteria developed for a software design course: Information 
Technology (IT), Oral Communication (OC), Problem Solving (PS), Value and Professionalism (VP), and 
Creativity and Innovation (Cl). The contents of each assessment criterion are: 

IT—including four sub-criteria: A student is proficient in instructions and functions of the animation design 
software, “Adobe® Flash CS3,” is able to integrate other Adobe® software that allows editing web pages, 
pictures, videos, and so on, can configure a design to appear in its entirety on a personal website, accepts the 
Internet as an important tool for accessing information pertinent to design. 

OC—including two sub-criteria: A student is comfortable using any mode or media of communication for 
delineating software design ideas, especially, demonstrating skill with technology for effective 
communication. 

PS—including three sub-criteria: A student can correctly debug the software from error codes that appear during 
the process of software design (the most important one), is able to conduct immediate tests on the results of 
different animation software designs, can revise developed software according to principles of design. 

VP—including two sub-criteria: A student can attend class on time as well as demonstrate an active attitude 
toward the process of learning (i.e., maintain an interest in software design), and is willingly undertakes 
software development according to the principles of design. 

Cl—including two sub-criteria: A student can use different methods to solve problems arising in the process of 
software design, complete a design that is instinctively interesting to users. 

RESEARCH METHODS 

The research methods implemented in this study include: Study’s framework, experimental design, including 
study procedures, participants, and data collection, and analysis methods. 

Study Framework 

Reflective thinking is an important strategy in students’ learning processes. According to the literature, for 
enhancing problem-solving abilities as well as increasing creativity, students should receive frequent 
encouragement to think reflectively when engaging in the process of learning or designing software. Thus, this 
study examines the effects of reflective thinking on students’ learning performances during software design 
processes. The assumption is that for software design assignments, lower achieving students will demonstrate 
learning-process improvement after thinking reflectively and achieve performance comparable to higher 
achieving students. 

A further assumption is that all students’ computer self-efficacy significantly improves after acquiring skills in a 
software design course. After practicing reflective thinking in the process of design, especially, among those 
deemed to have lower achieving learning performances at the beginning of the software design course, 
self-efficacy for computer use should be equivalent to those students demonstrating higher achievement. This 
study adopts the Compeau and Higgins (1995) questionnaire, which has 10 questions answerable by “YES” or 
“NO.” If the answer is the former, the students can make a personal self-assessment of strength of 
self-confidence on a 1 to 10 scale. All the students completed the questionnaire twice, one before the beginning 
of the course and again at the course’s completion. The operatational assumption is that students demonstrating 
low performance in the software design course could acquire abilities for software design and increased their 
levels of self-efficacy for computers by thinking reflectively. Figure 1 illustrates this study’s framework. Two 
proposals for null hypotheses are: 

Null Hypothesis 1 - A reflective thinking strategy cannot aid lower achieving students to reach learning 
performance comparable with higher achieving students in a software design course. 

Null Hypothesis 2 - The level of computer self-efficacy after the course has no significant incensement 
compared with that before the course for students in the lower achieving group. 



Figure 1: Study framework. 


Copyright © The Turkish Online Journal of Educational Technology 


91 






TOJET: The Turkish Online Journal of Educational Technology - April 2012, volume 11 Issue 2 


Experimental Design and Study Procedures 

In this study, all students had the same educational background and the same starting-point of skill level and 
familiarity for the “Windows Software Design” course in spring 2008. After learning and practicing the windows 
software design about four weeks, grouping the students produced to categories (Figure 2), control and 
experiment groups based on their average scores from mid-semester quizzes. The quizzes evaluated graphic 
design, video clipping, program design, and schedule arrangement. Five multi-dimensional assessment criteria, 
information technology (IT), oral communication (OC), problem solving (PS), value and professionalism (VP), 
and creativity and innovation (Cl), with different percentage weights evaluated students’ performances (Table 1). 



Figure 2: Experimental design. 


Table 1: Weight for each criteria 


Criteria 

Quiz 

Overall Scores 

IT 

75% 

45% 

OC 

- 

16% 

PS 

12% 

13% 

VP 

- 

19% 

Cl 

13% 

7% 

Total 

100% 

100% 


The experimental group consisted of students whose quiz scores were lower than the average score of all the 
students. These lower achieving students received an additional assignment, in the processes of their learning, of 
writing a weekly reflection diary of their weblogs and their animated software designs until they completed their 
projects. The control group’s requirement was merely to finish the assigned software design assignment before 
the end of the course. Evaluations of two groups of students’ overall learning performances included scores for 
three kinds of design scores from quizzes, homework, one midterm exam, one final exam, a design work, and 
oral and written presentations for the design work. All the evaluative components received different weights in 
percentage by using the same criteria as used in the quizzes. All the students (i.e., study’s participants) 
completed the Compeau and Higgins (1995) Computer Self-efficacy Questionnaire before and after the course to 
determine if their levels of self-efficacy for computers had increased. Specifically, the self-efficacy levels of 
those who were originally regarded as lower achieving students could be observed as well after the practices of 
reflective thinking during their windows software design. 

In addition, this study includes a qualitative research approach component. The participants analyzed the 
designated dynamic website (http://www.acmcf.org.tw/model/) before and after the course through computer 
generated, written documents, submitted the course instructor. This website contains many animated effects 
similar to the software design assignments completed for the design course. An analysis of the participants’ 
perspectives of the dynamic website, before and after the course explored whether or not the students attended to 
design principles and/or considered the design concepts. 

Participants 

Among the sophomores in the college who studied the course, “Windows Software Design,” 13 students were 
willing to participate in this study. Each participant had basic ability to create web page, understood 
object-oriented concepts of program design, and showed interest for learning animation software design. At the 
beginning of the course, all participants completed the Computer Self-Efficacy Questionnaire and received four 
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weeks of instruction for designing animation. The content covered an introduction to Adobe® Flash CS3 design 
tools, animated text and graphic design techniques, animated button and movie-clip design techniques, 
commonly used Adobe® Flash CS3 action scripts, and an introduction to usability and accessibility 
considerations in design. Based on their performances from quizzes related to animation design during the first 
four weeks, two groups categorized the students. The experimental group consisted of seven students whose total 
scores on the quizzes were lower than the average score of all the students. These students comprised the lower 
achieving group for this study, since their quizzes indicated difficulty comprehending design principles or 
spending less time on animation design practice. Oppositely, the other six students comprised the control group. 
These higher achieving students often attended the computer lab to practice animated design, resulting in more 
facility for completing quizzes competently. 

Data Collection and Analysis Methods 

Two evaluative criteria from students’ scores were the bases for dividing students into groups for this study 
experimental design: One criteria was the average quiz score; the other was the students’ overall score for the 
semester, including quizzes, homework, midterm exam, final exam, design work, and oral and written 
presentations for the design work. Each score ranged from 0 to 100, according to the five multi-dimensional, 
assessment criteria: IT, OC, PS, VP and Cl. Based on the goals established according to the course syllabus, each 
criterion score had a specific weight, as shown in Table 1. 

As a result, the control group students all scored higher than the average quiz scores obtained by all participant 
students. The quizzes primarily measured students’ IT, PS and Cl abilities. Contrarily, the experimental group’s 
students scored lower than the overall average of quiz scores. Consequently, these lower achieving students 
received encouragement to write reflective diaries of their weblog activities during the design processes. 
Students in the experimental group wrote reflective diaries, and knew that the activity entailed no extra-credit 
points. 

Calculation of students’ overall scores was according to the weight of each score of the five multi-dimensional 
assessment criteria. An assessment of the scores of the two groups of students for each criterion ensued, and 
descriptive statistics analysis of the two groups of students produced average overall scores and the standard 
deviation. An Independent-Sample T-Test compared the two groups of students’ overall scores and the scores in 
each multi-dimensional assessment criterion to determine the presence of a significant difference. In addition,, 
before and after the course, the two groups of students completed a pre-test and post-test Computer Self-Efficacy 
Questionnaire. Establishing the questionnaire’s content validity employed the Compeau & Higgins (1995) 
Computer Self-efficacy Questionnaire instead of the GSE or other self-efficacy scales, not specifically for 
predicting students’ computing behaviors. Reliability analysis, followed by a Paired-Samples T-Test, explored 
the presence of significant differences between pre-test and post-test average scores. 

In addition to analyzing quantitative data, such as students’ quiz results, overall learning performance scores, and 
scores for computer self-efficacy, students’ comments of the designated dynamic website underwent two types 
of qualitative analyses: The first, literal data from students’ evaluations of the designated dynamic website, 
before and after the course, consisted of written files of the analyzed contents. A calculation of the number of 
words written by each student, analyzed via T-Test, determined if the numbers of words significantly increased 
from the initial evaluation to the second evaluation. The other qualitative analysis used software (Nvivo 7) to 
evaluate the written contents, to determine if the students’ assessments had a basis in principles of design, or if 
any design concepts had considered when they gave comments on the designated dynamic website. 

RESEARCH RESULTS 

The average score of all the participating students’ scores (M) on quizzes was 77.37 (standard deviation, 
SD=11.60). The experimental group’s students scored lower than the control group students (experimental group: 
M=70.64; SD=11.82; control group: M=85.20; SD=4.37). Also, the difference between the two groups reached a 
significant level (T=2.84; p= .02). The bases for study’s results are: First, the descriptive analysis results of the 
two groups of students’ learning performances including the overall scores the students received in the software 
design course and their scores evaluated by five multi-dimensional assessment criteria, in percentage, with 
different weights; second a comparative analysis of the students’ pre- and post-test scores for computer 
self-efficacy, and third, the results of students’ written evaluations of the contents of the designated dynamic 
website from before and after the course. 

Students ’ Learning Performances 

Immediately subsequent to the quizzes, the experimental group’s students adopted a method of learning through 
reflective thinking of the whole process of software design. Table 2 shows the descriptive analysis results of the 
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two groups of students’ scores for each score item and the overall scores. As a result, after verification through 
Paired-Sample T-Test, the difference in each score item or overall score between them appears insignificant at a 
significance level of 0.05 (Table 3). 

In addition, based on multi-dimensional assessment criteria, the performance of experimental group’s students 
underwent further analysis. Table 4 reports the descriptive analysis results of the scores of the two groups of 
students for the five multi-dimensional assessment criteria. By conducting Paired-Sample T-Test, no significant 
difference exists for any criterion at a significance level of 0.05 (Table 5). Notably, the scores of experimental 
group’s students for OC and VP were close to the scores of control group’s students. In other words, the oral 
abilities (OC scores) for presenting software design ideas from students in the experimental group may have 
improved along with their value and professionalism (VP scores) in software design. 

Table 2: Descriptive analysis results of the students’ overall and each item scores 


Score Item/Group 

Experimental group 

Control group 

M 

SD 

M 

SD 

Homework 

68.38 

7.76 

77.56 

10.85 

Midterm exam 

69.57 

10.96 

78.25 

9.03 

Final exam 

76.07 

7.87 

79.58 

15.59 

Design work 

80.86 

4.10 

80.33 

4.23 

Oral presentation 
for the design work 

73.57 

32.75 

85.00 

4.47 

Written presentation 
for the design work 

84.29 

6.65 

85.00 

6.99 

Overall Score 

75.58 

5.72 

80.86 

3.62 


Table 3: T-Test results of the students’ learning performances 


Item/ Test 

T value 

Degrees of 
freedom 

Significance 

Homework 

1.40 

5 

0.22 

Midterm exam 

1.08 

5 

0.33 

Final exam 

0.62 

5 

0.56 

Design work 

0.16 

5 

0.88 

Oral presentation 
for the design work 

0.85 

5 

0.44 

Written presentation 
for the design work 

0.56 

5 

0.60 

Overall Score 

1.95 

11 

0.08 


Table 4: Descriptive analysis results of the students’ scores in each criterion 


Item/Group 

Experimental group 

Control group 

M 

SD 

M 

SD 

IT 

73.98 

6.53 

79.86 

4.07 

OC 

80.15 

8.94 

84.18 

4.79 

PS 

74.94 

5.29 

80.65 

5.93 

VP 

83.75 

4.82 

85.66 

8.39 

Cl 

74.23 

4.90 

79.57 

4.25 


Table 5: T-Test results of the students’ learning performance in each criterion 


Item/ Test 

T value 

Degrees 

freedom 

Significance 

IT 

1.91 

11 

0.08 

OC 

0.99 

11 

0.35 

PS 

1.84 

11 

0.09 

VP 

0.51 

11 

0.62 

Cl 

2.08 

11 

0.06 


The study’s results do not support Null Hypothesis HI. The students in the experimental group, those having 
lower achievement in the first four weeks of the course, received encouragement thinking reflectively in the 
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process of software design to achieve comparable scores to those in the control group for each item and for 
overall scores. The experimental groups performance for each multi-dimensional assessment criterion, especially 
Cl (creativity and innovation), appears to be competent in comparison to the students in the control group. 

Students ’ Computer Self-Efficacy Test Scores 

This study also explores the participating students’ changes in their self-efficacy for computers, especially those 
students in the lower achieving group. All students completed the same questionnaire before and after the course. 
Comparisons of responses determined the existence of any significant differences. Then, a comparison of the two 
groups of students’ pre-test or post-test of self-efficacy for computers determined whether or not they attained 
the same levels of self-efficacy at the end of the software design course. One student in the control group did not 
answer all the questions completely, causing exclusion of this particular data. Overall, the reliability analysis 
results of the questionnaire before and after the course reached 0.70 and 0.92 respectively (Note: the content 
validity had been established). Table 6 shows the pre-test and post-test average scores and the standard deviation 
(M and SD) of the two groups of students’ self-efficacy for computers (total: 12 persons,7 in the experimental 
group and 5 in the control group) as well as the differences in the students’ pre-test and post-test scores, as 
analyzed by T-Test. 

As a result, the control group students’ self-efficacy for computers pre- and post-test scores was higher than the 
scores from the experimental group’s students. However, no significant difference appeared between the two 
groups of students for either of test scores. To be specific, the experimental group students’ self-efficacy for 
computers, for either the pre-test or post-test, neither demonstrated a significant difference at a significance level 
of 0.05. Hence, Null Hypothesis (H2) gains support. Notably, the students in both the control and the 
experimental groups scored lower for the post-test than for the pre-test. This result shows that the students’ 
degree of self-confidence for using packaged software declined by the end of the course. In terms of using 
Adobe® and Flash CS3 as well as other series of Adobe® software for editing pictures, videos, web pages, and 
so on, the students had lower self-confidence than before the course began. Even the experimental group’s 
students who adopted learning through reflective thinking demonstrated a lower degree of self-confidence for 
using packaged software. Further discussion of this aspect appears later. 


Table 6: Students’ self-efficacy with computers 


Group/ Pre-test and 

Pre-test 

Post-test 

T-Test 

post-test 

M SD 

M SD 

(pre- and post-) 

Experimental group 
Control group 

All the students 

6.70 .74 

7.34 .64 

7.14 .69 

6.63 1.11 

7.24 1.08 

6.89 1.09 

Not significant 

Not significant 

Not significant 

T-Test 

(group comparison) 

Not significant 

Not significant 

— 


Students ’ Analytical Contents of the Designated Dynamic Website 

Table 7 shows the pre-test and post-test average numbers of words written by all participating students— 181 
words and 230 words for each group respectively. However, the experimental group students wrote more than 
the control group students did, irrespective of pre-test or post-test timing, but none reached a significance level. 
Therefore, further analyses of the two groups of students’ written contents may be worthwhile. 


Table 7: Comparison of the average number of words for the analyses of the website 


Group/ Pre-test and 
post-test 

Number of words 
in pre-test 

Number of words 

T-Test 

in post-test 

Experimental group 

197 

231 

Not significant 

Control group 

176 

230 

Not significant 

All the students 

181 

230 

Not significant 

T-Test 

Not significant 

Not significant 

— 


As for the pre-test analytic words provided by the two groups of students, all expressed personal assessments of 
the contents of the dynamic website but no concrete suggestions for changes. However, most of the students 
were able to list the dynamic website’s advantages and disadvantages in their post-test analyses, and their 
awareness of layout designs, word types, colors, etc. For example, in the pre-test, one student in the experimental 
group said: 
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From the animation on the website, we are able to grasp the general content from the very 
beginning, no need to click every button. It also has much stuff, providing many links for us to 
click and browse, unlike the general web pages that show the contents directly and make the 
layouts so chaotic. 

In the post-test, the same student wrote: 

The blank space at the side seems to be wasted. It should have more animated effects added. I have 
no comments about the change of photo on the main page, but a part of something there seems to 
be cut off. If we click the button “simulation,” we can see the word “simulation” written there. 
Some of the clicked pages are divided into two areas. It will be more convenient for us to know 
what the location address is, and we will not feel like turning it off upon seeing it. 

However, by reading the pre-test words written by the control group students, apparently the principles of design 
had more frequent mention in their written contents, and the control group seemed more capable of pondering 
the kinds of designs that tend to cause users to learn the contents of the website more easily. Two comments are 
particularly telling: 

The contents of all the displayed stuff are shown on the left side, which is very convenient for 
users to click and see what they want to browse. If a user has to read English contents on the main 
page, the following contents should be written in English, too. But, I think that it may be a better 
way to have both English and Chinese versions for users to choose on any page. 

The main page on this website is not as complicated as those of other websites... But, as for the 
introduction, if the language version we have chosen is in both English and Chinese, the typeface 
should not be too small. If only English, the size of the typeface will be a little small. The website 
has an advantage—it has not only the literal introduction regarding collections, but also provides 
animation-simulated videos, so that the viewers can understand its operation through another way. 

The post-test words written by the student of the control groups were of greater value for reference: 

The page design of this website is useful and direct. We can quickly link to the contents we intend 
to see. As for its design of the arts, no outstanding performance was shown. The literal and graphic 
introduction plus the impressive video files will enable users to obtain rich content right here on 
this website. 

In terms of color, I’ve discovered that this website has applied many gray colors to its backgrounds 
and buttons, so that the entire page seems to be somewhat dark. As for its layout and writing, the 
typeface used in this website seems to be PMingLiU, but the effect is not good. Maybe it will be 
better if it is changed to be BiauKai. As for animation, I think that the website did a great job. 
Many pictures are hard to draw. It is so incredible to make the animation well. 

As a whole, even though the experimental group students wrote pre-test and post-test analyses of the dynamic 
website’s design in greater volume than the control group students, the control group students were able to grasp 
the key points of design in the pre-test. To the contrary, the students in experimental group had to undergo the 
process of learning through reflective thinking, to enhance their abilities to design and analyze, and hence the 
post-test literal contents written by them shows an equivalent analysis to that written by the students in the 
control group. 

CONCLUSIONS AND DISCUSSIONS 

The purpose of this study is to examine the effects of reflective thinking, applied to software design processes, 
on students’ learning performances. This study shows that learning through reflective thinking can result in great 
influence on lower achieving students who have learning obstacles for designing software, and reflective 
thinking has significance for their of software design processes. The effects not only showed in the performance 
in each scored item (e.g., final exam, design work), but also displayed changes in those students’ learning 
performances from multi-dimensional assessment criteria, specifically OC and VP, and qualitative literal data. 
After the course, the lower achieving students’ written analyses of the designated dynamic website also showed 
that they had obtained deeper understanding of the principles of a site’s design. The evidence of their deep 
understanding arises drawn from the quality of their written analyses, in concert with the study’s approach, 
adopted from Chang and Chou (2011) whom asserted a strong correlation between achievement and reflection 
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quality. In another words, due to learning through reflective thinking, the lower achieving students are likely to 
become aware of design principles, become able to design software by following them, complete the animation 
software design assignment and express personal design concepts in an orderly manner when orally presenting 
their designs. However, this study finds that, overall, students represent a low degree of self-confidence when 
using software to design animations after the end of the course. This finding is not surprising, since only the 
students having strong self-efficacy are willing to confront new challenges with optimism; whereas, the students 
having weak perceptions of self-efficacy are likely to be less confident in new, stressful situations (Schwarzer, 
1997). Another caution arises from the scale used in this study to determine the students’ levels of self-efficacy; 
the scale is sensitive to, and situation-dependent upon, students’ current computer use status. Also, prior studies 
indicated that various internal and external factors, such as computer experience, gender, grade levels, task 
complexity and so on, can affect levels of perceptions of self-efficacy tinman & (,’clikli. 2009; Topkaya, 2010). 

Finally, this study’s results may provide instructors with an alternative approach of teaching: That is, reflective 
thinking, which can enhance students’ creativity and other abilities. The evaluation method mentioned in this 
study may serve as a reference for teachers’ lesson planning. In addition, in terms of the changes in students’ 
perceptions of self-efficacy with computers, future studies might explore the necessity of re-designing the 
questionnaire to be more suitable for application to exploring students’ use for designing animation software, 
instead of using the questionnaire for determining the degree of self-confidence in using common packaged 
software. Or, future studies may explore how reflective thinking can influence a student’s perception of 
self-efficacy with computers and the influences from other internal and external factors, such as time for 
practicing designs in the computer classroom, personal computer equipment, and the frequency of browsing 
models of designs. 
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